Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracing of setup and teardown actions in JUnit 4 #8030

Conversation

daniel-mohedano
Copy link
Contributor

@daniel-mohedano daniel-mohedano commented Nov 28, 2024

What Does This Do

  • Implements tracing of setup and teardown actions available in JUnit 4 (@Before, @After, @BeforeClass, @AfterClass, @BeforeParam, @AfterParam).

Motivation

To provide more complete information regarding test runs, the setup and teardown actions can be traced. This is already implemented for JUnit 5, which implies that it would also be nice to have for JUnit 4.

Additional Notes

Contributor Checklist

Jira ticket: SDTEST-1224

@pr-commenter
Copy link

pr-commenter bot commented Nov 28, 2024

Benchmarks

Startup

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-29T11:41:13 2024-11-29T11:48:14
git_branch master daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4
git_commit_date 1732876820 1732879953
git_commit_sha 9f01834 948fe8a
release_version 1.44.0-SNAPSHOT~9f01834ab7 1.44.0-SNAPSHOT~948fe8ac30
start_time 2024-11-29T11:40:59 2024-11-29T11:48:01
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732881248 1732881248
ci_job_id 723438389 723438389
ci_pipeline_id 50141202 50141202
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.357 ms) : 1337, 1376
.   : milestone, 1357,
appsec (1.778 ms) : 1755, 1801
.   : milestone, 1778,
appsec_no_iast (1.767 ms) : 1742, 1792
.   : milestone, 1767,
iast (1.504 ms) : 1481, 1527
.   : milestone, 1504,
profiling (1.561 ms) : 1537, 1585
.   : milestone, 1561,
tracing (1.512 ms) : 1487, 1536
.   : milestone, 1512,
section candidate
no_agent (1.353 ms) : 1334, 1372
.   : milestone, 1353,
appsec (1.764 ms) : 1740, 1789
.   : milestone, 1764,
appsec_no_iast (1.77 ms) : 1745, 1794
.   : milestone, 1770,
iast (1.491 ms) : 1468, 1514
.   : milestone, 1491,
profiling (1.514 ms) : 1490, 1538
.   : milestone, 1514,
tracing (1.519 ms) : 1494, 1544
.   : milestone, 1519,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.357 ms [1.337 ms, 1.376 ms] -
appsec 1.778 ms [1.755 ms, 1.801 ms] 421.323 µs (31.1%)
appsec_no_iast 1.767 ms [1.742 ms, 1.792 ms] 410.053 µs (30.2%)
iast 1.504 ms [1.481 ms, 1.527 ms] 147.444 µs (10.9%)
profiling 1.561 ms [1.537 ms, 1.585 ms] 204.183 µs (15.0%)
tracing 1.512 ms [1.487 ms, 1.536 ms] 154.958 µs (11.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.334 ms, 1.372 ms] -
appsec 1.764 ms [1.74 ms, 1.789 ms] 411.549 µs (30.4%)
appsec_no_iast 1.77 ms [1.745 ms, 1.794 ms] 416.781 µs (30.8%)
iast 1.491 ms [1.468 ms, 1.514 ms] 138.115 µs (10.2%)
profiling 1.514 ms [1.49 ms, 1.538 ms] 161.456 µs (11.9%)
tracing 1.519 ms [1.494 ms, 1.544 ms] 166.149 µs (12.3%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (378.368 µs) : 359, 398
.   : milestone, 378,
iast (492.647 µs) : 471, 514
.   : milestone, 493,
iast_FULL (655.985 µs) : 634, 678
.   : milestone, 656,
iast_GLOBAL (519.947 µs) : 499, 541
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (495.839 µs) : 475, 517
.   : milestone, 496,
iast_INACTIVE (451.754 µs) : 431, 472
.   : milestone, 452,
iast_TELEMETRY_OFF (489.516 µs) : 468, 511
.   : milestone, 490,
tracing (448.227 µs) : 428, 469
.   : milestone, 448,
section candidate
no_agent (380.47 µs) : 361, 400
.   : milestone, 380,
iast (494.585 µs) : 473, 516
.   : milestone, 495,
iast_FULL (660.218 µs) : 639, 682
.   : milestone, 660,
iast_GLOBAL (520.383 µs) : 499, 542
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (497.827 µs) : 476, 519
.   : milestone, 498,
iast_INACTIVE (453.714 µs) : 433, 474
.   : milestone, 454,
iast_TELEMETRY_OFF (479.762 µs) : 459, 501
.   : milestone, 480,
tracing (454.786 µs) : 434, 476
.   : milestone, 455,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 378.368 µs [358.682 µs, 398.053 µs] -
iast 492.647 µs [471.36 µs, 513.934 µs] 114.279 µs (30.2%)
iast_FULL 655.985 µs [634.372 µs, 677.598 µs] 277.617 µs (73.4%)
iast_GLOBAL 519.947 µs [498.778 µs, 541.116 µs] 141.579 µs (37.4%)
iast_HARDCODED_SECRET_DISABLED 495.839 µs [474.587 µs, 517.09 µs] 117.471 µs (31.0%)
iast_INACTIVE 451.754 µs [431.219 µs, 472.29 µs] 73.387 µs (19.4%)
iast_TELEMETRY_OFF 489.516 µs [467.667 µs, 511.365 µs] 111.148 µs (29.4%)
tracing 448.227 µs [427.775 µs, 468.678 µs] 69.859 µs (18.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.47 µs [360.716 µs, 400.224 µs] -
iast 494.585 µs [472.997 µs, 516.173 µs] 114.115 µs (30.0%)
iast_FULL 660.218 µs [638.762 µs, 681.674 µs] 279.748 µs (73.5%)
iast_GLOBAL 520.383 µs [498.97 µs, 541.795 µs] 139.913 µs (36.8%)
iast_HARDCODED_SECRET_DISABLED 497.827 µs [476.233 µs, 519.422 µs] 117.358 µs (30.8%)
iast_INACTIVE 453.714 µs [432.983 µs, 474.444 µs] 73.244 µs (19.3%)
iast_TELEMETRY_OFF 479.762 µs [458.846 µs, 500.678 µs] 99.292 µs (26.1%)
tracing 454.786 µs [433.773 µs, 475.798 µs] 74.316 µs (19.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4
git_commit_date 1732876820 1732879953
git_commit_sha 9f01834 948fe8a
release_version 1.44.0-SNAPSHOT~9f01834ab7 1.44.0-SNAPSHOT~948fe8ac30
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1732881900 1732881900
ci_job_id 723438390 723438390
ci_pipeline_id 50141202 50141202
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.398 s) : 15398000, 15398000
.   : milestone, 15398000,
appsec (15.0 s) : 15000000, 15000000
.   : milestone, 15000000,
iast (18.99 s) : 18990000, 18990000
.   : milestone, 18990000,
iast_GLOBAL (18.209 s) : 18209000, 18209000
.   : milestone, 18209000,
profiling (14.869 s) : 14869000, 14869000
.   : milestone, 14869000,
tracing (15.048 s) : 15048000, 15048000
.   : milestone, 15048000,
section candidate
no_agent (15.41 s) : 15410000, 15410000
.   : milestone, 15410000,
appsec (14.946 s) : 14946000, 14946000
.   : milestone, 14946000,
iast (18.36 s) : 18360000, 18360000
.   : milestone, 18360000,
iast_GLOBAL (17.692 s) : 17692000, 17692000
.   : milestone, 17692000,
profiling (15.258 s) : 15258000, 15258000
.   : milestone, 15258000,
tracing (14.933 s) : 14933000, 14933000
.   : milestone, 14933000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.398 s [15.398 s, 15.398 s] -
appsec 15.0 s [15.0 s, 15.0 s] -398.0 ms (-2.6%)
iast 18.99 s [18.99 s, 18.99 s] 3.592 s (23.3%)
iast_GLOBAL 18.209 s [18.209 s, 18.209 s] 2.811 s (18.3%)
profiling 14.869 s [14.869 s, 14.869 s] -529.0 ms (-3.4%)
tracing 15.048 s [15.048 s, 15.048 s] -350.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.41 s [15.41 s, 15.41 s] -
appsec 14.946 s [14.946 s, 14.946 s] -464.0 ms (-3.0%)
iast 18.36 s [18.36 s, 18.36 s] 2.95 s (19.1%)
iast_GLOBAL 17.692 s [17.692 s, 17.692 s] 2.282 s (14.8%)
profiling 15.258 s [15.258 s, 15.258 s] -152.0 ms (-1.0%)
tracing 14.933 s [14.933 s, 14.933 s] -477.0 ms (-3.1%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.44.0-SNAPSHOT~948fe8ac30, baseline=1.44.0-SNAPSHOT~9f01834ab7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.339 ms) : 2297, 2380
.   : milestone, 2339,
iast (2.078 ms) : 2025, 2130
.   : milestone, 2078,
iast_GLOBAL (2.118 ms) : 2065, 2171
.   : milestone, 2118,
profiling (1.951 ms) : 1909, 1993
.   : milestone, 1951,
tracing (1.919 ms) : 1879, 1959
.   : milestone, 1919,
section candidate
no_agent (1.465 ms) : 1453, 1476
.   : milestone, 1465,
appsec (2.333 ms) : 2291, 2375
.   : milestone, 2333,
iast (2.076 ms) : 2023, 2128
.   : milestone, 2076,
iast_GLOBAL (2.118 ms) : 2065, 2170
.   : milestone, 2118,
profiling (1.954 ms) : 1911, 1998
.   : milestone, 1954,
tracing (1.917 ms) : 1878, 1957
.   : milestone, 1917,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.339 ms [2.297 ms, 2.38 ms] 871.869 µs (59.4%)
iast 2.078 ms [2.025 ms, 2.13 ms] 610.9 µs (41.7%)
iast_GLOBAL 2.118 ms [2.065 ms, 2.171 ms] 651.145 µs (44.4%)
profiling 1.951 ms [1.909 ms, 1.993 ms] 484.71 µs (33.0%)
tracing 1.919 ms [1.879 ms, 1.959 ms] 452.412 µs (30.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.453 ms, 1.476 ms] -
appsec 2.333 ms [2.291 ms, 2.375 ms] 868.645 µs (59.3%)
iast 2.076 ms [2.023 ms, 2.128 ms] 611.137 µs (41.7%)
iast_GLOBAL 2.118 ms [2.065 ms, 2.17 ms] 653.324 µs (44.6%)
profiling 1.954 ms [1.911 ms, 1.998 ms] 489.733 µs (33.4%)
tracing 1.917 ms [1.878 ms, 1.957 ms] 452.758 µs (30.9%)

@daniel-mohedano daniel-mohedano added the comp: ci visibility Continuous Integration Visibility label Nov 29, 2024
@daniel-mohedano daniel-mohedano marked this pull request as ready for review December 2, 2024 10:25
@daniel-mohedano daniel-mohedano requested review from a team as code owners December 2, 2024 10:25
@daniel-mohedano daniel-mohedano merged commit cdee1c8 into master Dec 2, 2024
103 checks passed
@daniel-mohedano daniel-mohedano deleted the daniel.mohedano/SDTEST-1224-setup-teardown-tracing-junit4 branch December 2, 2024 11:04
@github-actions github-actions bot added this to the 1.44.0 milestone Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp: ci visibility Continuous Integration Visibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants